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Summary 


• The Student Transcript-Enhanced Placement Study (STEPS) helps colleges analyze the 
association between students' high school performance in English and math and their 
achievement in college English and math courses to inform discussions about practical 
ways to improve college placement policy and practices. 

• Previous studies indicate that the use of information contained on high school 
transcripts may be a viable option to improve our current placement process. 

• Long Beach City College (LBCC) developed data management and analysis infrastructure 
useful to other colleges analyzing high school transcript data. 

• Grassroots efforts lead to the improvement of this infrastructure. 

• Many colleges encountered data availability and data quality issues. 

• Ten of the original twenty colleges were able to complete their analyses. 

o High school test scores associated most strongly with college placement in 
English and math. 

o High school grades associated most strongly with college grades for English, 
o Predicting college math grades is complex given the greater number of levels and 
test types in high school. 

• Overall, these findings suggest that high school transcript data are at least partially 
predictive of college performance and could be used for a variety of purposes: 

o As a part of the multiple measures component of existing community college 
placement systems 

o As the primary measure for placement of recent high school students 
o To measure the impact of curriculum alignment efforts 
o To create advising guides for high school students 
o To inform broader policies at regional or state levels 

• Other implications include the importance of data quality, research capacity, 
anticipating costs of policy changes, and the potential benefits of aligning high school 
and college curricula. 

• Further research on a statewide dataset is currently underway. 
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Introduction 


The first day of school can be exhilarating with its blend of anticipation and anxiety. Recent high 
school graduates entering a California community college for the first time often confront an 
additional burden—assessment results that tell them they have to repeat an English or math 
class already completed at the secondary level. As John Hetts, Director of Institutional Research 
for Long Beach City College often notes in his presentations, telling incoming students just out 
of high school that "you're not ready for college" can be discouraging. Of particular concern is 
the fact that while our community colleges place the "remedial" label on these students, it may 
be our own placement processes that are in need of remediation. 

When critically examining California community college placement systems and K-12 curricula, 
we may ask how aligned are California community college English and math courses to 
California K-12 standards? To what extent do the K-12 standards prepare students for the core 
English composition and college-level math courses that are articulated among California public 
community colleges and universities? How effective are our assessments in informing us about 
students' readiness for college-level work? Of course, some students do need remediation. 
Some did not have a successful high school experience. Others graduated long ago and their 
academic skills have faded with time. However, when we consider the academic preparation of 
recent high school graduates from California who earned passing grades or higher in their high 
school courses, we may be underestimating their ability to succeed in college-level work-in 
part due to the underutilization of their high school course and test data for advising and 
placement. 

In the context of these questions, a group of colleges, foundations, a system office and 
researchers have come together to explore responses and strategies to improve student 
progression through our educational systems through the Student Transcript-Enhanced 
Placement Study (STEPS) 1 . This report documents the process and current state of STEPS, 
findings and implications discovered to date, and planned continuing efforts and will inform a 
companion practitioner brief for the field. This document begins with historical background and 
related efforts before turning to detailed and technical descriptions of the work in which 
colleges engaged. The technical sections are useful for researchers wishing to replicate or 
extend the work described but can be skipped by readers more interested in the final section 
with findings, implications, and future activities. 


1 http://www.rpgroup.org/projects/STEPS 
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What is the STEPS Project? 


The RP Group—with funding from The Rosalinde and Arthur Gilbert Foundation, the Walter S. 
Johnson Foundation and LearningWorks and with support from the California Partnership for 
Achieving Student Success (Cal-PASS) and the California Community College Chancellor's Office 
(CCCCO)—has been conducting a pilot project providing technical assistance on data 
management and analysis to a group of invited California community colleges who wish to 
enhance their placement and advising systems for recent local high school graduates by 
utilizing high school transcript information. This effort, called the Student Transcript-Enhanced 
Placement Study (STEPS) began in early 2012 and focuses on helping colleges replicate analyses 
conducted by Cal-PASS and Long Beach City College (LBCC) examining the association between 
students' high school performance in English and math and their achievement in college English 
and math courses, respectively. STEPS analyses are intended to drive discussions about 
practical ways to improve college placement policy and practices. 

Why are Multiple Measures for Assessment and Placement Important? 

Historically, California community colleges have emphasized placement testing as an important 
component of addressing basic skills, also referred to as developmental or remedial education. 
The assumption is that accurate placement would allow for students with similar skill levels to 
be in the same course and receive the instruction most appropriate for advancing them to the 
next level in an English or math sequence. Over the years, each community college has created 
its own placement system under matriculation regulations that guide placement system 
development and validation (Matriculation Handbook 2011). These regulations were initially 
created by the Matriculation Act passed in 1988 and legally prevent incoming students from 
initially enrolling in higher-level English or math courses unless they earn a sufficient score on a 
placement test or have successfully completed the prerequisite course at the college. Concerns 
about and challenges to this approach emerged immediately. A suit (Romero-Frias et at. v. 
Mertes et ai, 1988) brought by the Mexican American Legal Defense and Education Fund 
(MALDEF) contested the California community college assessment and prerequisite process, 
suggesting it was biased and unfair and that it ignored other information such as prior 
coursework at other schools and colleges (Wiseley, 2009). The suit was dropped after a 
successful negotiation between the California Community College Chancellor's Office (CCCCO) 
and MALDEF that resulted in a number of changes to matriculation practices including a 
requirement that community colleges not use a single test score for placement: 

Multiple measures ensure that no single test score or assessment measure is used to 
place students into courses. Course placement on the basis of a test score by itself is 
prohibited [Title 5, § 55521(a)(3)] because a test score alone is not as informative and 
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valid for placement purposes as when applying it with another measure. (Matriculation 
Handbook 2011, page 2.5) 

These "multiple measures" can consist of additional standardized tests, performance 
assessments such as essays and interviews, surveys of students' self-efficacy, prior coursework, 
and other background information. A recent survey of California community colleges found that 
the strong majority of respondents (81%) reported using high school educational background 
information as a multiple measure (REL WestEd 2011). Of those, most (over 70%) reported 
using surveys to collect high school information while only about a quarter to a third used 
transcripts to verify high school background data. 

Despite the changes to assessment and placement policies and practices, concerns about the 
efficacy of these key matriculation services persist. Test scores can seem to overshadow these 
additional measures; as stated by one student participating in a related study about assessment 
and placement in California's community colleges, "You spent four years in your high school, 
and they're judging your four years just off of that test. It puts you backwards" (Venezia, 

Bracco, & Nodine, 2010). 

In addition, many students do not seem to appreciate the consequences of the placements 
until after they receive their placement test scores and realize that they cannot take college- 
level courses until after one or more terms of preparatory work (Venezia, Bracco, & Nodine 
2010). Students often do not prepare for these tests and may not perceive the placement 
process as connected to their high school coursework or college registration and matriculation. 
Moreover, an unintended consequence of the multiple measures requirement has been to 
enhance the variability in college placement systems. While there are only a handful of 
commercial assessments and a minority of institutions develops their own tests 2 , colleges 
generally set their own cut scores, use a wide range of multiple measures, employ numerous 
methods for collecting and verifying background information, and place variable emphases on 
test scores versus multiple measures (WestEd REL, 2011). Another measure used by many 
California community colleges is the Early Assessment Program (EAP) 3 lead by the California 
State University (CSU) system in partnership with the California Department of Education (CDE). 
This program consists of an additional set of question items on the California Standards Tests 
(CST) administered to 11 th grade students in the subjects of 11 th grade English and Algebra II or 
Summative Math for students enrolled in Pre-Calculus. Students receive a score indicating if 
they are "ready for college" at a CSU. Those receiving "not ready for college" feedback can then 
access a series of support efforts through EAP to achieve readiness for CSU by graduation. 
Currently, 76 California community colleges accept either English or math EAP scores for 


2 http://extranet.cccco.edU/Portals/l/SSSP/Matriculation/Assessment/ApprovedlnstrumentsApril2009RevSept2011.pdf 

3 http://www.calstate.edu/eap/ 
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placement 4 with some accepting only one subject or the other and some accepting both English 
and math EAP results. While the number of high school students taking the EAP test has been 
growing, the majority of students do not take the math EAP as a student has to be enrolled in a 
specific level of math in 11 th grade in order to be eligible to take the assessment. 

Multiple measures and traditional placement tests are not the only approaches California 
community colleges are using for placing recent local high school graduates. There are different 
local and regional approaches throughout the state as high schools and colleges partner to 
experiment with different approaches. For example, Grossmont College, the Grossmont Union 
High School District (GUHSD), San Diego State University, and local elementary and middle 
schools with support from Cal-PASS created the English Curriculum Alignment Project (ECAP). 
This multi-year effort saw faculty from elementary through university level meet and align their 
English sequences. In fall 2009 and 2010, a pilot was conducted to allow students earning a 
grade of A or B in 12 th grade English to enroll directly in college level composition at the 
community college. The fall 2009 pilot cohort had an 83% success rate (grade of C or better) 
compared to 70% for comparison group students and 67% for all other composition students 
(Linfor, Harris, & Jendian, 2011). The fall 2010 pilot cohort had an 86% success rate compared 
to 68% for other composition students. These early successes have been followed by an 
expansion of the pilot to more students. This model represents an alternative approach to 
placement testing for incoming high school graduates in the form of directly articulating high 
school coursework with college coursework. The amount of time invested by faculty to do the 
necessary alignment work is substantial, but is counterbalanced by cost savings in a reduced 
number of testing units that must be purchased and less demand for developmental courses. 

The great variation in California community college placement systems can create an additional 
challenge for students who move among campuses. While some colleges honor each other's 
placement recommendations, it is not consistent or required. Not being able to transport 
placement results from one campus to another creates a barrier for students enrolling at more 
than one college and also incentivizes "placement shopping," where students take placement 
tests at several different colleges to achieve a higher placement. To help create more 
consistency for students, the Student Success Task Force 5 recommendations call for the 
implementation of a statewide common assessment. In addition, there are efforts to create a 
statewide centralized repository of assessment data to help placement scores be more portable 
and to support validation research. Further, there is a growing body of evidence outlined below 
that detailed high school test and course performance data can provide a valuable 
enhancement to current multiple measure practices. 


4 http://extranet.cccco.edu/Divisions/StudentServices/EAP/AcceptingEAP.aspx 

5 http://californiacommunitvcolleges.cccco.edu/PolicyinAction/StudentSuccessTaskForce.aspx 
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How Useful Might High School Data Be for College Placement? 

Several studies indicate that the use of information contained on high school transcripts, such 
as grade point average (GPA) and standardized test scores and grades in math and English 
courses, may be a viable option to improve our current placement process (Belfield & Crosta, 
2012; Jaffe, 2012; Brown & Conley, 2007; Brown & Niemi, 2007; Willett, Hayward, & Dahlstrom, 
2008; Wurtz, 2008). However, these findings have not been incorporated into placement policy 
or practice on a large scale in California. In 2011, the Long Beach City College (LBCC) 
institutional research office replicated and extended one of these earlier studies (Willett, 
Hayward, & Dahlstrom, 2008) on their campus (Fuenmayor, Hetts & Rothstein, 2011). LBCC 
researchers used data from the California Partnership for Achieving Student Success (CalPASS) 
intersegmental data system 6 to link high school transcript data from high schools in LBCC's 
largest feeder school district. Long Beach Unified School District (LBUSD), to LBCC course 
records. 

The LBCC researchers used high school data including the highest level of high school English 
and math completed, scores on the California Standards Test (CST), and overall grades. College 
variables included level of first English or math course attempted and grade received in that 
course. Using regression analyses to assess the predictive power of high school information, 
they found significant associations between high school and college performance in English and 
math. LBCC researchers presented their findings to the LBCC community, which lead to a series 
of discussions about how to improve the college's placement process. In turn, the institution 
launched a pilot effort called Promise Pathways 7 that included placing recent LBUSD graduates 
based upon their high school English and math performance, which resulted in more students 
being placed into transfer-level English and math coursework when enrolling at LBCC. 
Guaranteed space in core courses incentivized students to attend full-time and to take English 
and math in their first year. 

The first semester of the pilot showed a significant increase in the number of students taking 
and successfully completing transfer-level English and math (Hetts and Willett, 2012). Prior to 
the pilot, 13% of LBUSD graduates placed into transfer-level English in 2011 while 60% of LBUSD 
graduates entered transfer-level English under the Promise Pathways initiative in fall 2012. 
Despite the much greater number of students enrolling in transfer-level English, success rates 
(earning a grade of C or better) held steady with 64% of former LBUSD students in 2011 being 
successful compared to 62% of the Promise Pathways students in 2012. For math, 9% of former 
LBUSD students in 2011 placed into transfer level math compared to 30% of 2012 Promise 


6 CalPASS is a voluntary data system of student level transcript data with encrypted identifiers to enhance security and privacy. 
More information on this system can be found at www.calpass.org . 

7 http://lbcc.edu/promisepathways/ 


Student Transcript-Enhanced Placement Study (STEPS) Progress Report - August 2013 - www.rpgroup.org 6 







Pathway students. Again, despite tripling the number of students in transfer-level math, success 
rates remained relatively consistent with a 55% success rate for former LBUSD students in 2011 
and a 51% success rate for Promise Pathway students in 2012. These results suggest that a very 
large number of recent high school graduates had been underplaced and in fact were prepared 
to engage in transfer-level work. These findings have fueled the continuation of the pilot and 
movement towards institutionalization. A key factor that primed this pilot was a long-standing 
collaborative effort called the Long Beach Promise that, among other activities, facilitated 
coordinated curricula alignment between K-12 and postsecondary education segments in the 
Long Beach area. Further, while exploring the utility of high school transcripts in college place 
drove the initial discussions, the actual policies put into place included a broad array of 
strategies. Given that LBCC implemented a suite of changes simultaneously, it would be difficult 
to impossible to isolate the impact of any single alteration in policy or practice. 

LBCC Creates the Foundation of STEPS 

In the process of conducting their analyses, the LBCC research office developed infrastructure 
that would prove useful to other colleges wishing to replicate this study. In particular, they 
developed a Microsoft Access module that automated the importation of CalPASS Plus data 
files into a data mart and created a file for analysis in the SPSS statistical program. They also 
developed a series of programming scripts in SPSS to conduct the regression analyses. Finally, 
they created extensive documentation that allows others to replicate their analyses and 
generously shared their work with the field. The infrastructure LBCC created provided the 
foundation for the statewide pilot replication study described in this report. 

STEPS Statewide Pilot Research Questions and Implementation Technical Details 

About twenty colleges were invited to participate in STEPS with resources from the Rosalinde 
and Arthur Gilbert Foundation, the Walter S. Johnson Foundation and LearningWorks and with 
support Cal-PASS and the California Community College Chancellor's Office (CCCCO). The RP 
Group provided technical assistance to colleges to download, process, and analyze CalPASS 
data from their own campus and local feeder high schools. The primary research questions 
driving the analyses were: 

1. What is the relationship between high school performance measures both in general 
and specifically for English and the level of the first English course attempted at the 
community college? 

2. What is the relationship between high school performance measures both in general 
and specifically for English and success in the first English course attempted at the 
community college? 
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3. What is the relationship between high school performance measures both in general 
and specifically for math and the level of the first math course attempted at the 
community college? 

4. What is the relationship between high school performance measures both in general 
and specifically for math and success in the first math course attempted at the 
community college? 

The effort also explored data availability and data quality issues. In addition, opportunities for 
and barriers to making improvements in placement practices received consideration. 

STEPS 1.0 - Research Limitations and Lessons Learned. During the early stages of the process, 
the first institutions to engage in the pilot provided feedback on the extent to which the 
Microsoft Access module and SPSS scripts developed for LBUSD and LBCC would function with 
data from their own schools and colleges. Some of the earliest analyses were hampered by a 
variety of issues. One challenge was inaccurate or ambiguous course coding primarily with high 
school courses. These courses are coded according to the California Basic Educational Data 
System (CBEDS) schema 8 . Originally designed for mapping faculty assignments, CBEDS codes 
were then applied to courses and used not only to distinguish different disciplines but also to 
map sequences within a discipline such as English or math. In the absence of clear and accurate 
CBEDS codes, these sequences have to be derived from course titles and other information that 
requires much more time and introduces a source of error in cases where a course title does 
not clearly indicate the level of a course. 

Another barrier to the analysis consisted of instances where an insufficient number of years of 
data from a high school to adequately track students through high school and into college. 
While California's community colleges have a centralized data system (the Chancellor's Office 
Management Information System or "COMIS") with data from all institutions that dates back to 
1994, the analogous K-12 system (California Pupil Achievement Data System or "CalPADS") is 
relatively new and still in development. The Cal-PASS data system receives voluntary 
submissions from K-12 districts that vary greatly in their capacity to submit multiple years of 
data or any data at all. In some cases, these challenges are due to information technology 
resource limitations or technical issues such as recent changes in information systems that 
make historical data difficult to access. In addition, some K-12 districts have missing data 
elements for some years. For example, a district may have five years of enrollment and grade 
data but only two years of testing data submitted to the Cal-PASS system. In cases where there 
are only a few hundred or fewer student records and/or several years of data missing for 


s http://www.cde.ca.gov/ds/sp/cl/documents/assncodemap2010.doc 
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certain data elements, regression models can suffer and institutions struggle to draw 
conclusions from the results. 

A third major constraint discovered through the early statewide pilot was the ability of a 
community college research office to commit sufficient time to the analyses, especially in the 
face of challenges with data availability and quality. These barriers resulted in about half of the 
colleges not being able to participate fully in the study. 

STEPS 2.0—Improvements to the Transcript Analysis Methodology. At the same time, these 
experiences also inspired a grassroots effort to improve the efficiency of data screening and 
analysis required for the transcript analysis initiative. This effort—lead by Dan Lamoree (Mount 
San Antonio College) with input and assistance from the LBCC research team, Dustin Tamashiro 
(Pasadena City College) and RP researchers—a "STEPS 2.0" Microsoft Access module was 
released. This new module underwent field testing among a subset of the pilot colleges leading 
to further refinements. Some of the significant improvements included: 

1) A more generalized structure to better apply to any school and college 

2) Incorporation of all file manipulation into the Access module, leaving SPSS scripts solely 
for performing statistical functions 

3) A user-friendly screen to review and easily correct course codings 

4) Expanded inclusion criteria to capture a larger number of students 

5) Incorporation of additional variables of interest such as time between high school and 
college English and math courses 

6) Additional documentation including a training video 

While the 2.0 version began as a grassroots effort among the research community, the RP 
Group staff formally incorporated these enhancements into the pilot project and supported 
Dan Lamoree's efforts. To accompany the module, the RP Group staff created a new set of SPSS 
scripts that included LBCC's regressions, alternate forms of the regressions to accommodate 
colleges that had more limited software licenses, and two additional analyses. The first analysis 
was a set of bivariate correlations (both parametric and non-parametric) to help those with a 
small number of cases and/or a large number of missing data elements. While the regressions 
consider all variables simultaneously, bivariate correlations compare only two variables at a 
time and allow for interpretation of the more robust data elements. The drawback to this 
approach is that sometimes simple two-way associations can be misleading if there are other 
variables with a strong influence. For example, correlating a high school grade point with a 
college grade point alone does not account for the level of the course taken in college. 

The second analysis created a set of two tables that cross the last English or math course taken 
in high school with the first English or math class respectively attempted in college. One table 


Student Transcript-Enhanced Placement Study (STEPS) Progress Report - August 2013 - www.rpgroup.org 9 




populates the cells with row percentages to assess the level to which students are transitioning 
in college in relation to the highest level they completed in high school. The second table 
populates the cells with college success rates (grade of C- or better) to assess the achievement 
of students in the college course attempted. These tables were intended to provide researchers 
with an intuitive display of student transition and achievement for a broad audience (see 
example table for math in Appendix A). 

The release of STEPS 2.0 was accompanied by webinars and presentations at the RP Group 
researcher conference. User feedback indicated that the new version significantly reduced data 
processing time while increasing accuracy through more management screening and correction 
tools. Some of the earliest participants reran their analyses to include the new students in the 
expanded inclusion criteria and apply some of the changes made to the regression modules. By 
the end of the analysis phase, a total often colleges were able to participate in addition to 
LBCC's efforts. 

What Did the Statewide Pilot Study Find? 

Eleven pilot colleges were able to complete regression analyses. Each college conducted four 
regressions to: 

• Predict level of first college English course attempted 

• Predict level of first college math course attempted 

• Predict success in first college English course attempted 

• Predict success in first college math course attempted 

Predictor variables included: 

• Most recent California Standards Test (CST) score in English (or math) (standardized) 

• Count of "A-G" or college preparatory courses completed 9 

• Level of most recent high school course in English (or math) (ordinal rank) 

• Grade point in most recent high school course in English (or math) 

• High school grade point average excluding English (or math) 

• Level of first college course attempted in English (or math) (ordinal rank, only used for 
predicting success in college course) 

High schools and colleges had differing numbers of levels of English and math offerings 
although three to four levels was common for English and five to eight levels was typical for 
math for both high schools and community colleges. Ordinal regressions were used to predict 


9 California has seven categories of high school courses to complete in order to be eligible for admission to a University of 
California or California State University. 

http://admission.universitvofcalifornia.edu/freshman/requirements/a-g-requirements/ 
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the level of college English or math for all 11 colleges. To predict success, eight colleges used 
logistic regression, two used ordinal regression, and one college was unable to conduct 
regressions for predicting success in English or math in the time frame of the pilot. Table 1 
shows the count of students in the regressions and regression strength indices. 

Table 1. Counts of colleges and students in the regression models with sample size weighted 
mean pseudo R z s. 


Count of Students Within Weighted mean 

Community College pseudo R 2 




Count of 
Colleges 

Minimum 

Maximum 

Average 

(Mean) 

Cox and 

Snell 

Nagelkerke 

College 

English level 

11 

128 

9,717 

1,844 

0.32 

0.36 

variable 

Math level 

11 

65 

9,717 

1,985 

0.11 

0.16 

predicted 

English success 

10 

65 

8,299 

1,362 

0.35 

0.37 


Math success 

10 

65 

8,299 

1,462 

0.13 

0.18 


The number of students ranged from 65 to nearly 10,000 with an average around 1,500 to 
2,000. One measure of strength of ordinal and logistic regression models is the pseudo R 2 (r 
squared). This index ranges from zero to one (asymptotically) where zero indicates the 
predictors have almost no predictive value and indices close to one indicate the predictors have 
almost perfect predictive value. There are several different pseudo R 2 values reported for 
ordinal and logistic regressions with the Cox and Snell and Nagelkerke being two of the most 
commonly reported. The pseudo R 2 values for predicting English level and success were 
moderately strong in the context of education research and suggest the models may include 
many of the most important predictor variables. The pseudo R 2 values for predicting math level 
and success were moderately weak in the context of education research and suggest the 
models could benefit from either additional variables or other changes to the models. 

One challenge in the math models results from the California Standards Test (CST) forms that 
relate directly to the type of math taken. For example, if a high school student's last math class 
was Algebra II in 11 th grade, they would take the Algebra II CST. If a high school students last 
math class was Pre-Calculus in 11 th grade, they would take the "summative math" CST. In 
contrast, all high school students in 11 th grade English take the same form of the English CST 
with the exception of students taking alternative test forms for special circumstances. One 
strategy to address these challenges in math is to conduct separate regressions for each 
different CST form. However, this approach requires a larger number of students. Another 
possible approach would be to include the CST form as an addition categorical variable or as a 
series of indicator variables. 
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To convey to a broad audience how the utility of different predictors varies among the colleges, 
a set of descriptive charts was created. In each chart, the horizontal axis shows predictor 
categories and the vertical axis indicates the number of colleges for which that predictor had a 
statistically significant association with the predicted outcome (level of or success in college 
English or math). The color of the bars indicates the strength of significant predictor variables 
with "warmer" colors (orange) being stronger predictors and "cooler" colors (green to blue) 
being weaker predictors. The strength of predictors was determined by an RP Group staff 
researcher inspecting regression coefficients within each college and assigning the coefficients 
to one of the three categories of strongest, intermediate, and weakest. This method involves 
human judgment and arbitrary categories and therefore the charts should be viewed as 
illustrative rather than as an objective meta-analysis. As described later in this report, a 
statewide analysis is in progress that will conduct a series of regressions among a large number 
of high schools and community colleges to more objectively discern the variability of predictor 
utility among educational institutions. 

When predicting level of college English (Figure 1): 

• The high school English CST predictor category is the most consistent and strongest as 
seen by the bar indicating that the variable was significant for all 11 colleges and the 
shading being orange for all colleges. 

• The count of A-G courses was significant for six colleges but was weak for all six of these 
colleges. 

• High school English course level was significant for only four colleges and was an 
intermediate predictor for most of those colleges. 

• High school grade in English was significant for only four colleges but was a weak 
predictor for most of those colleges. 

• High school GPA excluding English grades was significant for seven colleges and was an 
intermediate predictor for most of the colleges. 

For predicting college English level, the CSTs were clearly the most important predictor variable. 
As college English levels were generally determined by placement processes that typically 
weight placement test scores heavily, one could simplify this interpretation to "tests predict 
tests". While a concise statement, it should be cautioned that other variables had predictive 
value that varied among colleges. 
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Predicting College English Level 
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Courses Course Course 
Level Grade 


Predictor Variable Category 


* High school GPA excluding English grades 

Figure 1. Count of colleges with significant predictor variables for each predictor category and 
their predicting strength within each college for predicting level of college English. 

When predicting level of college math (Figure 2): 

• The high school math CST predictor category is the most consistent as seen by the bar 
indicating that variable was significant for all 11 colleges; however, the strength varied 
as seen by the variation in shading. 

• The count of A-G courses was significant for six colleges but was weak for most of the 
colleges. 

• High school math course level was significant for seven colleges and, unlike when 
predicting English level, was a strong predictor for most of these colleges. 

• High school grade in math was significant for eight colleges but was a weak predictor for 
all of those colleges. 

• High school GPA excluding math was significant for seven colleges and was an 
intermediate predictor for most of the colleges. 

For predicting college math level, the CSTs were clearly the most consistent predictor variable 
but the strength varied. As with predicting the level of college English, one could also state 
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"tests predicts tests" for math but the highest level of math completed in high school is also a 
powerful predictor for the majority of colleges in the pilot study. 


Predicting College Math Level 
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Courses Course Course 
Level Grade 


Predictor Variable Category 


* High school GPA excluding math grades 

Figure 2. Count of colleges with significant predictor variables for each predictor category and 
their predicting strength within each college for predicting level of college math. 

When predicting success in college English (Figure 3): 


• The high school English CST predictor category is the least consistent and weakest 
predictor being significant for only three colleges and was weak for all three. 

• The count of A-G courses was significant for seven colleges and was of intermediate 
strength for most colleges. 

• High school English course level was significant for five colleges and was an intermediate 
predictor for most of those colleges. 

• High school grade in English was significant for eight colleges but the predictive strength 
varied among colleges. 

• High school GPA excluding English grades was significant for nine colleges and was a 
strong predictor for all of those colleges. 
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The level of college English attempted was significant for six colleges and was a strong 
predictor for most colleges. 




In general, students attempting high-level courses experienced lower success rates. For 
predicting college English success, the non-English high school grades were clearly the most 
important predictor variable. This result suggests that general student skills rather than subject 
specific skills may be an important factor in college success. Grades in English also had fairly 
consistent predictive value. Another simplified interpretation could be "grades predicts 
grades". Again, it should be cautioned that other variables had predictive value that varied 
among colleges. 


Predicting College English Success 
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Weakest 
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Predictor Variable Category 


* High school GPA excluding English grades 

Figure 3. Count of colleges with significant predictor variables for each predictor category and 
their predicting strength within each college for predicting success in college English. 

When predicting success in college math (Figure 4): 

• The high school math CST predictor category is among the least consistent and weakest 
predictor being significant for five colleges and was weak for the majority of those 
colleges. 
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• The count of A-G courses was significant for only four colleges and was of intermediate 
strength for those colleges. 

• High school math course level was significant for seven colleges and was an 
intermediate predictor for most of those colleges. 

• High school grade in math was significant for six colleges and was of intermediate 
strength. 

• High school GPA excluding math grades was significant for seven colleges but the 
predictive strength varied among colleges. 

• The level of college math attempted was significant for seven colleges and was an 
intermediate predictor for most colleges. 

In general, students attempting higher level college courses experienced lower success rates. 

For predicting college math success, predictors had less consistency of utility among colleges 
than seen for English. Levels of math completed in high school and attempted in college 
appeared to be the most important predictors of success in college math. However, similar to 
predicting success in college English, the non-math grades were also among the most important 
predictor variables again suggesting that general student skills rather than subject specific skills 
may be an important factor in college success. The more complex situation for predicting math 
success qualifies the "grades predicts grades" statement, raising instead the more ungainly 
"predictors of college math success vary among high schools and colleges." One reason for this 
additional complexity relates back to the possible reason for the math regressions being weaker 
than English with the greater diversity of CST forms in math and more variability in the numbers 
of levels of math in high school and college. 
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Predicting College Math Success 
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Figure 4. Count of colleges with significant predictor variables for each predictor category and 
their predicting strength within each college for predicting success in college math. 

Overall, these findings suggest that high school transcript data are at least partially predictive of 
college performance. A key theme emerging from these results could be "more information on 
high school performance is better than less when placing students at college". 

Barriers to Implementation of the Use of High School Transcripts in Placement Processes 


After conducting these regressions, community college researchers shared the findings within 
their colleges and, in some cases, also with feeder high schools. College personnel engaged in 
discussions of the findings typically included discipline faculty, assessment staff, administrators, 
and researchers. These conversations resulted in the identification of a number of barriers to 
implementing transcript analysis in the placement process. 


Some colleges reporting feeling challenged to address the implications of these results in the 
face of barriers to changing practice and the demands of maintaining day-to-day operations. 
One key barrier to utilizing high school information in placement processes was the challenge of 
incorporating high school transcripts into current practices. While community colleges have 
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processes for generating transcripts, the open access nature of these institutions means that 
unlike selective universities, there are generally not processes in place to receive and process 
high school transcripts. Adding this capacity to community colleges is a significant task that 
requires a number of steps including developing agreements with feeder high schools and/or 
the parent of high school students and/or adult former high school students to transfer 
identifiable data, either electronically or in hard copy. Electronic transfers require information 
technology infrastructure while hard copy transfers require secure office space resulting in 
demands for facilities resources. In addition, training is needed and staff time allocated to these 
new tasks. For at least one college attempting to incorporate high school transcripts into their 
placement systems, a group of admissions and records staff had to be trained, reclassified and 
promoted in order to interpret transcripts. 

At the same time, no colleges reported strong opposition by any group of faculty, staff, or 
administrators to the general concept of using high school information in placement. The 
attitudes and experiences of the personnel involved would need to be monitored during a 
longer period of planning and implementation to fully assess if other concerns might arise. In 
general, it appears the most important short-term barriers to initiating the use of high school 
transcripts in college placement are one-time and ongoing facilities and personnel costs. 

Implications for Practice 

Importance of Quality Data and Capacity to Manage and Analyze Data 

This pilot has a series of implications for practice at both high schools and colleges. The early 
challenges with the limited extent of data submissions and accuracy of some high school course 
data underline the fundamental importance of accurate and complete data for the planning, 
implementation, and evaluation of using high school transcripts in college placement. The data 
management and analysis infrastructure created for the project reduced the costs associated 
with conducting analyses. However, it was apparent that many colleges lacked the capacity to 
conduct the analyses. Capacity in this case is not necessarily the skill or ability to manage and 
analyze the data but rather the time available to perform this work in addition to the tasks 
already required by the institution. One skill area that was developed for many of the 
community college researchers was awareness and knowledge of CBEDS code and English and 
math offerings in general at their local feeder high schools. 

Unexpected Costs 

A second implication is the recognition that such an implementation can have unexpected costs 
such as changes in job descriptions, training, new equipment acquisitions, or expanded office 
space requirements. These costs are in addition to more readily anticipated costs such as the 
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need for participation of high school and college personnel, students, parents, and the broader 
community in collaborative discussions of proposed changes and the development of 
agreements and protocols. 

Potential Benefit of Aligning Curricula 

A third key implication is the potential benefit of aligning English and math curricula among 
high schools and community colleges. Aligned curricula would allow students to progress 
forward in their English and math sequences after transitioning from high school to college. 
Increases in alignment of sequences between high school and college would be expected to 
increase the predictive validity of high school information. Curricular alignment that achieves 
the level of virtual articulation between high school and college would obviate the need for 
placement tests for recent high school students. However, placement systems would still be 
needed for out-of-state students who were educated under a different framework of standard 
or re-entry college students whose high school coursework may be dated. Exactly how long a 
high school course remains a sufficient prerequisite for the next level of coursework in college 
is unclear. Expiration dates for prior coursework should be established based on clear evidence 
and may vary widely among individuals complicating efforts to create simple, uniform policies 
on recency requirements. While the basic principle of aligned curricula appears simple, it 
requires significant effort to establish and maintain. K-12 curricula in the past have been guided 
by state standards and are currently shifting to meet new Common Core State Standards 10 . 
Community colleges have to meet articulation agreements for transfer-level English and math 
courses with universities. Within each of these shifting frameworks, discipline faculty from high 
schools and college have to commit substantial time and effort to determine how to align 
sequences and then establish on-going collaborative work to maintain alignment in the face of 
changes in standards, articulation agreements, and other factors. To help the alignment effort 
and assess its effectiveness, utilizing high school transcripts as a multiple measure in current 
community college placement systems will allow for the collection of data for monitoring and 
research as institutions increase the weight given to high school transcripts over time. 

Possible Uses of High School Transcript Data 

The associations between high school and college performance could be used for a variety of 
purposes: 

• As a part of the multiple measures component of existing community college placement 

systems 

• As the primary measure for placement of recent high school students 


10 For information on California's implementation, see http://www.cde.ca.gov/re/cc/ . 
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• To measure the impact of curriculum alignment efforts 

• To create advising guides for high school students 

• To inform broader policies at regional or state levels 

The items in this list are not mutually exclusive or exhaustive. Findings from STEPS effort cannot 
suggest a single most effective strategy or change in practice or policy as curriculum, 
placement, and advising are composed of many complex issues most of which were not directly 
addressed by STEPS. Each region will have to engage with their data and determine the best 
course of action for their students and community. 

Current Developments 

Common Core 

There are several current developments that have impacts on these efforts. One previously 
mentioned impending change is California's adoption of the Common Core State Standards 11 . 
This comes not only with new standards but new tests to assess students on these standards. 
College placement systems that begin to utilize high school transcripts will have to prepare for 
validating their systems with a new set of K-12 tests and grades based on a different set of 
standards. The Common Core promises updated standards for K-12 and greater consistency 
among states choosing to adopt that framework. This change could result in K-12 students who 
move among Common Core states experiencing a more consistent education. Further, 

California community colleges that align their curricula with the Common Core would 
theoretically be aligning not just within California but also with other states utilizing the 
Common Core framework. 

Student Success and Support Programs 

For the California community colleges, one change includes the Chancellor's Office renaming 
the processes and services bundled under of the name of Matriculation to Student Success and 
Support Programs. Beyond a name change, this signals shifts in priorities and strategies and 
may impact placement procedures and resources. In addition, two longstanding members of 
the Matriculation Advisory Committee, Douglass Glasnapp and John Poggio, who developed the 
placement test guidelines currently in use, are retiring from the committee. Their role will be 
replaced by the Buros Center for Testing who may update the guidelines. The Chancellor's 
Office is also developing a statewide multiple measures assessment warehouse that promises 
to lower the burden on colleges wishing to expand their multiple measures to include high 
school transcripts by providing centralized access to student information for placement. 


11 Mike Kirst provides a brief history of California standards and policy implications of the common core: 
http://www.edpolicvinca.org/publications/common-core-meets-state-policv-changes-almost-everything 
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advising and research. This assessment warehouse is connected to the Chancellor's Office plan 
to design and implement a common set of placement tests for all California community 
colleges. These centralization efforts are intended to create more consistency and portability of 
placement results for students. 

Accelerated Developmental English and Math 

Some faculty, such as Katie Hern of Chabot College and Myra Snell of Los Medanos College, 
have called for substantial changes in remedial English and math course sequences to reduce 
the number of levels students must progress through to reach transfer level. The argument is 
that each level of a sequence creates an opportunity for a student to drop out and that a 
shortened pathway will help "accelerate" students to college level coursework. One such effort 
led by Hern and Snell is the California Acceleration Project (CAP) 12 . A key to the CAP effort is 
identifying the core content in English and math required for success in transfer level (or college 
level) coursework. Outdated or peripheral content is identified and de-prioritized to streamline 
and focus the remedial courses on preparing students for either college composition or 
statistics, as appropriate. Because the accelerated math sequence focuses on a pathway to 
statistics, students who are preparing for science, technology, engineering, or math (STEM) 
majors would be expected to take additional math coursework. One accelerated math program, 
developed by the Carnegie Foundation for the Advancement of Teaching, includes two 
accelerated curricula; one is for statistics preparation while the other is for liberal arts math 
preparation 13 . 

CAP is currently being evaluated by the RP Group at a group of over 15 colleges that implement 
accelerated curricula in the 2011-2012 academic year. In addition to the question of the impact 
on student outcomes for those in the accelerated courses versus the traditional sequence, 
questions remain as to how student placement would be impacted. 

One possible consequence is that placement could be used solely to determine whether a 
student is ready for transfer-level work. Students who are not ready would take a single 
preparatory course as opposed to potentially facing two or more semesters of remediation. 

This change could potentially simplify placement decision systems as there would be only two 
levels into which to place students. Another possibility is for colleges to have both accelerated 
and non-accelerated curricula thereby increasing the complexity of placement and advising but 
providing more options for students from a variety of backgrounds. The next several years will 


12 http://cap.3csn.org/ 

13 http://www.carnegiefoundation.org/developmental-math 
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reveal how accelerated curricula efforts perform, how they are perceived, and how they are 
incorporated into colleges' English and math offerings. 

Northern California STEPS 

A group of community colleges in the far northern reaches of California have formed a research 
alliance to conduct STEPS analyses at their colleges. One of the colleges involved was in the 
initial STEPS pilot but encountered technical barriers during the time frame of the study. The 
colleges in this sparsely populated region are providing support to each other to share 
problems, solutions, and findings to accomplish the analyses and participate in regional 
discussions of how to improve placement systems. The Northern California STEPS group 
represents yet another grassroots effort coalescing around the issue of improving placement 
for students at California community colleges. These colleges are receiving technical assistance 
from Cal-PASS and the RP Group with Dan Lamoree providing the direct support on the STEPS 
2.0 Microsoft Access module. 

Statewide STEPS Study 

Currently, an effort is underway between the RP Group and Cal-PASS Plus to conduct an 
expanded analysis of the relationship between high school and community college 
performance. This study will include a large number of schools and colleges that were unable to 
participate in the pilot. The project will add additional variables and explore questions that 
arose in the pilot phase such as the impact on college performance of 1) the time elapsed from 
when students take English and math in high school and when they take it in college, 2) 
students' EAP scores, and 3) various student characteristics. Results of this study will be of 
particular use to colleges unable to participate in the STEPS pilot. Findings will also be used to 
inform the development of the statewide multiple measures assessment warehouse by 
suggesting data elements of the greatest value and creating a baseline placement algorithm 
that colleges can utilize or modify for their placement systems. 

Conclusion 

The STEPS effort revealed the intense interest and motivation on the part of a large number of 
college personnel to improve community college placement systems. This work resulted in a 
refined infrastructure lowering the burden for those wishing to conduct STEPS research at their 
campus. The findings from the pilot colleges suggest that high school information can be useful 
in college placement but that local validation studies are important to capture variation among 
student populations and curricula and to generate locally appropriate strategies. 
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Appendix A. Sample High School to Community College Math Transition Tables Using Artificial 
Data. 





First level attempted in community college 


Total 




Pre-AIg 

Beg Alg 

Int Alg 

Stats 

Pre-Calc 

Calc 

Percent 

Count 

Highest 

Pre-AIg 

67% 

33% 

0% 

0% 

0% 

0% 

100% 

30 

level 

Algebra 1 

25% 

50% 

25% 

0% 

0% 

0% 

100% 

80 

successfully 

Geometry 

25% 

50% 

25% 

0% 

0% 

0% 

100% 

40 

completed 

Algebra II 

10% 

19% 

38% 

19% 

10% 

5% 

100% 

210 

in high 

Pre-Calc 

8% 

17% 

25% 

25% 

17% 

8% 

100% 

120 

school 

Calculus 

0% 

0% 

20% 

20% 

40% 

20% 

100% 

50 

Total 

Percent 

15% 

25% 

28% 

15% 

11% 

6% 

100% 


Count 

80 

130 

150 

80 

60 

30 


530 




Success in first level attempted in community college 
Pre-AIg Beg Alg Int Alg Stats Pre-Calc Calc 

Total 

Highest 

Pre-AIg 

60% 

50% 

na 

na 

na 

na 

57% 

level 

Algebra 1 

65% 

60% 

55% 

na 

na 

na 

60% 

successfully 

Geometry 

65% 

60% 

55% 

na 

na 

na 

60% 

completed 

Algebra II 

70% 

65% 

55% 

80% 

65% 

60% 

64% 

in high 

Pre-Calc 

75% 

70% 

60% 

85% 

75% 

70% 

73% 

school 

Calculus 

na 

na 

60% 

70% 

75% 

65% 

69% 


Total 

66% 

62% 

56% 

81% 

72% 

65% 

65% 


Notes: Geometry is not shown in the community college sequence as it is generally not required to progress in the 
math sequence. 


Key to Abbreviations 

na = Not Applicable 
Pre-AIg = Pre-Algebra 

Beg Alg = Beginning Algebra (note this is similar to Algebra I in high school) 

Int Alg = Intermediate Algebra (note this is similar to Algebra II in high school) 
Stats = Statistics and other non-science major math such as Finite Math 
Pre-Calc = Pre-Calculus 
Calc = Calculus 
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